Submitted by: Sujit Thakur

Introduction to Computer Vision: Plant Seedlings Classification

Problem Statement

Defining Problem Statement ( As asked in grading rubic)

I (Sujit) , the assigned data scientist for this project, my task is to explore and analyze the provided dataset containing images of plant seedlings. The objective is to develop a predictive model that can accurately classify the species of these plants. By leveraging my expertise in data science, I aim to assist the organization in automating the process of plant identification, ultimately improving efficiency and productivity in the agricultural sector.

Context

In recent times, the field of agriculture has been in urgent need of modernizing, since the amount of manual work people need to put in to check if plants are growing correctly is still highly extensive. Despite several advances in agricultural technology, people working in the agricultural industry still need to have the ability to sort and recognize different plants and weeds, which takes a lot of time and effort in the long term. The potential is ripe for this trillion-dollar industry to be greatly impacted by technological innovations that cut down on the requirement for manual labor, and this is where Artificial Intelligence can actually benefit the workers in this field, as the time and energy required to identify plant seedlings will be greatly shortened by the use of AI and Deep Learning. The ability to do so far more efficiently and even more effectively than experienced manual labor, could lead to better crop yields, the freeing up of human inolvement for higher-order agricultural decision making, and in the long term will result in more sustainable environmental practices in agriculture as well.

Objective

The aim of this project is to Build a Convolutional Neural Netowrk to classify plant seedlings into their respective categories.

Data Dictionary

The Aarhus University Signal Processing group, in collaboration with the University of Southern Denmark, has recently released a dataset containing images of unique plants belonging to 12 different species.

• The dataset can be download from Olympus.
• The data file names are:
  • images.npy
  • Labels.csv

• Due to the large volume of data, the images were converted to the images.npy file and the labels are also put into Labels.csv, so that you can work on the data/project seamlessly without having to worry about the high data volume.

• The goal of the project is to create a classifier capable of determining a plant's species from an image.

List of Species

• Black-grass
• Charlock
• Cleavers
• Common Chickweed
• Common Wheat
• Fat Hen
• Loose Silky-bent
• Maize
• Scentless Mayweed
• Shepherds Purse
• Small-flowered Cranesbill
• Sugar beet

Note: Please use GPU runtime on Google Colab to execute the code faster.

Importing necessary libraries

# Installing the libraries with the specified version.
# uncomment and run the following line if Google Colab is being used
!pip install tensorflow==2.15.0 scikit-learn==1.2.2 seaborn==0.13.1 matplotlib==3.7.1 numpy==1.25.2 pandas==1.5.3 opencv-python==4.8.0.76 -q --user

import os
import numpy as np                                                                               # Importing numpy for Matrix Operations
import pandas as pd                                                                              # Importing pandas to read CSV files
import matplotlib.pyplot as plt                                                                  # Importting matplotlib for Plotting and visualizing images
import math                                                                                      # Importing math module to perform mathematical operations
import cv2                                                                                       # Importing openCV for image processing
import seaborn as sns                                                                            # Importing seaborn to plot graphs


# Tensorflow modules
import tensorflow as tf
from tensorflow.keras.preprocessing.image import ImageDataGenerator                              # Importing the ImageDataGenerator for data augmentation
from tensorflow.keras.models import Sequential                                                   # Importing the sequential module to define a sequential model
from tensorflow.keras.layers import Dense,Dropout,Flatten,Conv2D,MaxPooling2D,BatchNormalization # Defining all the layers to build our CNN Model
from tensorflow.keras.optimizers import Adam,SGD                                                 # Importing the optimizers which can be used in our model
from sklearn import preprocessing                                                                # Importing the preprocessing module to preprocess the data
from sklearn.model_selection import train_test_split                                             # Importing train_test_split function to split the data into train and test
from sklearn.metrics import confusion_matrix                                                     # Importing confusion_matrix to plot the confusion matrix
from sklearn.preprocessing import LabelBinarizer
# Display images using OpenCV
from google.colab.patches import cv2_imshow                                                      # Importing cv2_imshow from google.patches to display images
from sklearn.model_selection import train_test_split
from tensorflow.keras import backend
from keras.callbacks import ReduceLROnPlateau
import random
# Ignore warnings
import warnings
warnings.filterwarnings('ignore')

Note: After running the above cell, kindly restart the notebook kernel and run all cells sequentially from the start again.

Loading the dataset

# Mounting drive to get our data
from google.colab import drive
drive.mount('/content/drive')

Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount("/content/drive", force_remount=True).

## Checking location of our data file
!ls drive/MyDrive/Project\ CV/

 images.npy   Labels.csv  'project (1).html'

path_images = "drive/MyDrive/Project CV/images.npy"
path_labels = "drive/MyDrive/Project CV/Labels.csv"
# Load the image file of dataset
images = np.load(path_images)      #  code to read the dataset

# Load the labels file of dataset
labels = pd.read_csv(path_labels)  # code to read the dataset

Data Overview

Understand the shape of the dataset

## Printing shape of our data set
print(images.shape)         # code to check the shape
print(labels.shape)         # Code to check the shape

(4750, 128, 128, 3)
(4750, 1)

As you can see from above output that,

• For Images we have shape (4750,128,128,3) , that means we have 4750 images , which are of size 128*128 pixels and it has 3 color channel (RBG)

• For labels we can see that shape is (4750,1) which means we have a single column here which labels for 4750 to the relative image

Exploratory Data Analysis

• EDA is an important part of any project involving data.
• It is important to investigate and understand the data better before building a model with it.
• A few questions have been mentioned below which will help you understand the data better.
• A thorough analysis of the data, in addition to the questions mentioned below, should be done.

1. How are these different category plant images different from each other?
2. Is the dataset provided an imbalance? (Check with using bar plots)

Q1) How are these different category plant images different from each other?

def plot_images(images,labels):
  num_classes=10                                                                  # Number of Classes
  categories=np.unique(labels)
  keys=dict(labels['Label'])                                                      # Obtaing the unique classes from y_train
  rows = 3                                                                        # Defining number of rows=3
  cols = 4                                                                        # Defining number of columns=4
  fig = plt.figure(figsize=(10, 8))                                               # Defining the figure size to 10x8
  for i in range(cols):
      for j in range(rows):
          random_index = np.random.randint(0, len(labels))                        # Generating random indices from the data and plotting the images
          ax = fig.add_subplot(rows, cols, i * rows + j + 1)                      # Adding subplots with 3 rows and 4 columns
          ax.imshow(images[random_index, :])                                      # Plotting the image
          ax.set_title(keys[random_index])
  plt.show()
# Call the function
plot_images(images, labels)

Answer Q1) As we can see from above images we can see there is a lot of difference in each plant picture

we can see that the different category plant images different from each other in following attributes:

• Shape of leaf
• Size of leaf and plant
• Pose and Orientation of the species in the image
• Color brightness and contrast of the leaf
• Vein of the leaf
• Shape of tip of the leaf
• Midrib of the leaf

Q2) Is the dataset provided an imbalance?

import seaborn as sns
import matplotlib.pyplot as plt



# Plot the count of each class
plt.figure(figsize=(15, 5))  # Set the figure size to 15x5 inches
sns.countplot(x=labels['Label'])  # Create a count plot using seaborn
plt.xticks(rotation='vertical')  # Rotate the x-axis labels vertically

total_samples = len(labels)  # Total number of samples

for p in plt.gca().patches:  # Iterate over the bars in the plot
    count = p.get_height()  # Get the count value
    percent = count * 100 / total_samples  # Calculate the percentage
    plt.gca().annotate(f'{count:.0f} ({percent:.2f}%)',  # Add count and percentage on top of each bar
                       (p.get_x() + p.get_width() / 2., p.get_height()),
                       ha='center', va='center', xytext=(0, 10), textcoords='offset points')

plt.show()  # Display the plot

We can see from above bar plot that yes, the dataset provided is imbalanced data set, following is the descendingly sorted order of data category,

• Loosely Silky-bent with 654 count which makes 13.77% of data
• Common chickweed with 611 count which makes 12.86% of data
• Scentless Mayweed with 516 count which makes 10.86% of data
• Small- flowered Cranesbill with 496 count which 10.44% of data
• Fat hen with 475 count which makes 10% of data
• Charlock with 390 count which makes 8.21% of data
• Sugar beet with 385 count which makes up 8.11% of data
• Cleavers with 287 count which makes up 6.04% of data
• Black grass with 263which makes up 5.54 % of data
• Shephereds Purse with 231 cont which makes 4.86% of data
• Common wheat with 221 count which makes 4.65% of data
• Maize with 221 count which makes 4.65% of data

More and Additional EDA

Variation in classes

Lets check classes one by one and see in same class how do the different record/images differ from each other in following things

1. Shape and Size
2. Color
3. Pose and Angle of image

Note : We will check this on randomly picked 10 images from our dataset

import matplotlib.pyplot as plt
import numpy as np

def plot_class_images(images, labels, class_name, num_images=12):
    # Filter image names for the specified class
    class_indices = labels[labels['Label'] == class_name].index
    class_images = images[class_indices]

    # Check if there are enough images for the specified class
    if len(class_images) < num_images:
        print(f"Not enough images available for class {class_name}.")
        return

    # Randomly choose num_images images from the class
    chosen_indices = np.random.choice(len(class_images), num_images, replace=False)
    chosen_images = class_images[chosen_indices]

    # Set up the plot grid
    rows = 3
    cols = 4
    fig = plt.figure(figsize=(12, 8))

    # Plot images
    plt.title("Random chosen set for"+" "+ class_name )
    for i in range(num_images):
        ax = fig.add_subplot(rows, cols, i + 1)
        ax.imshow(chosen_images[i])
        ax.axis('off')  # Turn off axis
    plt.show()

EDA of Black-grass

plot_class_images(images, labels, class_name='Black-grass', num_images=12)

We can see for above images :

1. Shape and size = we can see some of the images like image of row 0 and column 0 has a very thin size and elongated shape , while some other images like row 2 and column 2 , we see zoomed image with diamond shape.

2. Color = we can see color of image depend on if the image is zoomed or not and on what is the concentrated on , like for zoomed in image like (2,2) we can see it is zoomed in but concentrated on leaf hence its dark green when compared to the image (2,3) where the it is zoomed in too a but concentrated on background of black pebbles and hence the species look to be light green

3. Pose and Angle : We can se that some images are taken from afar while some of them are taken from close or zoomed in like we can see difference in images like 0,0 and 2,2

EDA of Charlock

plot_class_images(images, labels, class_name='Charlock', num_images=12)

We can see for above images :

Shape and size = we can see some of the images like image of row 0 and column 0 has a approximately heart shape, while some other images like row 1 and column 1 , we see an elongated thick shape of brim of leaf.

Color = we can see color of image depend on focus of the image , as we can see some images like 1,1 are light in color while other images like 2,1 are dark green in color

Pose and Angle : We can see that atleast in pose and angle all images are mostlty taken from top

EDA of Cleavers

plot_class_images(images, labels, class_name='Cleavers', num_images=12)

We can see for above images :

Shape and size = we can see some of the images are short and thick while some other images are long and slim

Color = we can see color of image depend on focus of the image , as we can see some images are light in color while other images are dark green in color

Pose and Angle : We can see that atleast in pose and angle all images are mostlty taken from top

EDA for Common Chickweed

plot_class_images(images, labels, class_name='Common Chickweed', num_images=12)

We can see for above images :

Shape and size = we can see some of the leaf in images are short and thick while some other images are long and slim

Color = we can see color of image depend on focus of the image , as we can see some images are light green in color while other images are dark green in color

Pose and Angle : We can see that atleast in pose and angle all images are mostlty taken from top

EDA for Common Wheat

plot_class_images(images, labels, class_name='Common wheat', num_images=12)

We can see for above images :

Shape and size = we can see atleast in this type of species we can see that leaf in all images looks like slim and elongated

Color = we can see color of image depend on focus of the image , as we can see some images are light in color while other images are dark green in color

Pose and Angle : We can see that atleast in pose and angle all images are mostlty taken from top

EDA for Fat hen

plot_class_images(images, labels, class_name='Fat Hen', num_images=12)

We can see for above images :

Shape and size = we can see leaf looks similar in all randomly chosen images for this species

Color = we can see color of image depend on focus of the image , as we can see some images are light in color while other images are dark green in color

Pose and Angle : We can see that atleast in pose and angle all images are mostlty taken from top

EDA for Loosely Silky-bent

plot_class_images(images, labels, class_name='Loose Silky-bent', num_images=12)

We can see for above images :

Shape and size = we can see leaf looks similar in all randomly chosen images for this species

Color = we can see color of image depend on focus of the image , as we can see some images are light in color while other images are dark green in color

Pose and Angle : We can see that atleast in pose and angle all images are mostlty taken from top

EDA for Maize

plot_class_images(images, labels, class_name='Maize', num_images=12)

We can see for above images :

Shape and size = we can see leaf in some images has circular inner leaf (underdeveloped) and in some images its fully developed

Color = we can see color of image depend on focus of the image , as we can see some images are light in color while other images are dark green in color

Pose and Angle : We can see that atleast in pose and angle all images are mostlty taken from top

EDA for Scentless Mayweed

plot_class_images(images, labels, class_name='Scentless Mayweed', num_images=12)

We can see for above images :

Shape and size = we can see leaf in some images appear to be elonagted oval shape but in some it shows the leaf has some other leaf connected to itself as visible from image (0,2)

Color = we can see color of image depend on focus of the image , as we can see some images are light in color while other images are dark green in color

Pose and Angle : We can see that atleast in pose and angle all images are mostlty taken from top

EDA for Shepherds Purse

plot_class_images(images, labels, class_name='Shepherds Purse', num_images=12)

We can see for above images :

Shape and size = we can see leaf looks similar in all randomly chosen images for this species

Color = we can see color of image depend on focus of the image , as we can see some images are light in color while other images are dark green in color

Pose and Angle : We can see that atleast in pose and angle all images are mostlty taken from top

EDA for Small-flowered Cranesbill

plot_class_images(images, labels, class_name='Small-flowered Cranesbill', num_images=12)

We can see for above images :

Shape and size = we can see leaf in some images does differ in shape size

Color = we can see color of image depend on focus of the image , as we can see some images are light in color while other images are dark green in color

Pose and Angle : We can see that atleast in pose and angle all images are mostlty taken from top

EDA for Sugar beet

plot_class_images(images, labels, class_name='Sugar beet', num_images=12)

We can see for above images :

Shape and size = we can see leaf looks similar in all randomly chosen images for this species

Color = we can see color of image depend on focus of the image , as we can see some images are light in color while other images are dark green in color

Pose and Angle : We can see that atleast in pose and angle all images are mostlty taken from top

Data Pre-Processing

Convert the BGR images to RGB images.

for i in range(len(images)):
  images[i] = cv2.cvtColor(images[i], cv2.COLOR_BGR2RGB)        #  code to convert the images from BGR to RGB

Resize the images

As the size of the images is large, it may be computationally expensive to train on these larger images; therefore, it is preferable to reduce the image size from 128 to 64.

images_decreased=[]
height = 64                    # the code to define the height as 64
width =  64                    #  the code to define the width as 64
dimensions = (width, height)
for i in range(len(images)):
  images_decreased.append( cv2.resize(images[i], dimensions, interpolation=cv2.INTER_LINEAR))

Lets check images before and after resizing

## Before resizing
plt.imshow(images[3])

<matplotlib.image.AxesImage at 0x7d2f2f1684c0>

## After resizing
plt.imshow(images_decreased[3])

<matplotlib.image.AxesImage at 0x7d2f313ed390>

We can see that size of our image is reduced to 64*64

Data Preparation for Modeling

• Before you proceed to build a model, you need to split the data into train, test, and validation to be able to evaluate the model that you build on the train data
• You'll have to encode categorical features and scale the pixel values.
• You will build a model using the train data and then check its performance

Split the dataset

X_temp, X_test, y_temp, y_test = train_test_split(np.array(images_decreased),labels , test_size=0.1, random_state=42,stratify=labels)   # code to split the data with test_size as 0.1
X_train, X_val, y_train, y_val = train_test_split(X_temp,y_temp , test_size=0.1, random_state=42,stratify=y_temp)                       #  code to split the data with test_size as 0.1-temp

# code to check the shape of train, validation and test data
print(X_train.shape,y_train.shape)
print(X_val.shape,y_val.shape)
print(X_test.shape,y_test.shape)

(3847, 64, 64, 3) (3847, 1)
(428, 64, 64, 3) (428, 1)
(475, 64, 64, 3) (475, 1)

Encode the target labels

enc = LabelBinarizer()                                        # code to intialize the labelBinarizer
y_train_encoded = enc.fit_transform(y_train['Label'])
y_val_encoded=enc.transform(y_val['Label'])              # code to transform y_val
y_test_encoded=enc.transform(y_test['Label'])

y_train_encoded.shape,y_test_encoded.shape,y_val_encoded.shape    #  code to check the shape of train, validation and test data

((3847, 12), (475, 12), (428, 12))

Data Normalization

#  code to normalize the image pixels of train, test and validation data
X_train_normalized = X_train.astype('float32')/255.0
X_val_normalized = X_val.astype('float32')/255.0
X_test_normalized = X_test.astype('float32')/255.0

# Fixing the seed for random number generators
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)

Model Building

# Intializing a sequential model
model1 = Sequential()                             # Code to intialize a sequential model

# Code to add the first conv layer with 128 filters and kernel size 3x3 , padding 'same' provides the output size same as the input size
# Input_shape denotes input image dimension of images
model1.add(Conv2D(128, (3, 3), activation='relu', padding="same", input_shape=(64, 64, 3)))

# Code to add the max pooling to reduce the size of output of first conv layer
model1.add(MaxPooling2D((2, 2), padding = 'same'))

# Code to create two similar convolution and max-pooling layers activation = relu
model1.add(Conv2D(64, (3, 3), activation='relu', padding="same"))
model1.add(MaxPooling2D((2, 2), padding = 'same'))

model1.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model1.add(MaxPooling2D((2, 2), padding = 'same'))

# Code to flatten the output of the conv layer after max pooling to make it ready for creating dense connections
model1.add(Flatten())

# Code to add a fully connected dense layer with 16 neurons
model1.add(Dense(16, activation='relu'))
model1.add(Dropout(0.3))
# Code to add the output layer with 12 neurons and activation functions as softmax since this is a multi-class classification problem
model1.add(Dense(12, activation='softmax'))

# Code to use the Adam Optimizer
opt=Adam()
# Code to Compile the model using suitable metric for loss fucntion
model1.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

# Code to generate the summary of the model
model1.summary()

Model: "sequential_1"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d_3 (Conv2D)           (None, 64, 64, 128)       3584      
                                                                 
 max_pooling2d_3 (MaxPoolin  (None, 32, 32, 128)       0         
 g2D)                                                            
                                                                 
 conv2d_4 (Conv2D)           (None, 32, 32, 64)        73792     
                                                                 
 max_pooling2d_4 (MaxPoolin  (None, 16, 16, 64)        0         
 g2D)                                                            
                                                                 
 conv2d_5 (Conv2D)           (None, 16, 16, 32)        18464     
                                                                 
 max_pooling2d_5 (MaxPoolin  (None, 8, 8, 32)          0         
 g2D)                                                            
                                                                 
 flatten_1 (Flatten)         (None, 2048)              0         
                                                                 
 dense_2 (Dense)             (None, 16)                32784     
                                                                 
 dropout_1 (Dropout)         (None, 16)                0         
                                                                 
 dense_3 (Dense)             (None, 12)                204       
                                                                 
=================================================================
Total params: 128828 (503.23 KB)
Trainable params: 128828 (503.23 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________

Fitting model on train data

#  code to fit the model on train and also using the validation data for validation
history_1 = model1.fit(
            X_train_normalized, y_train_encoded,
            epochs=30,
            validation_data=(X_val_normalized,y_val_encoded),
            batch_size=32,
            verbose=2
)

Epoch 1/30
121/121 - 86s - loss: 2.4519 - accuracy: 0.1154 - val_loss: 2.4359 - val_accuracy: 0.1285 - 86s/epoch - 714ms/step
Epoch 2/30
121/121 - 86s - loss: 2.3423 - accuracy: 0.1864 - val_loss: 2.1047 - val_accuracy: 0.3248 - 86s/epoch - 707ms/step
Epoch 3/30
121/121 - 85s - loss: 2.0344 - accuracy: 0.3036 - val_loss: 1.7812 - val_accuracy: 0.3832 - 85s/epoch - 701ms/step
Epoch 4/30
121/121 - 87s - loss: 1.8897 - accuracy: 0.3410 - val_loss: 1.6992 - val_accuracy: 0.4486 - 87s/epoch - 718ms/step
Epoch 5/30
121/121 - 84s - loss: 1.7616 - accuracy: 0.3743 - val_loss: 1.5631 - val_accuracy: 0.4766 - 84s/epoch - 692ms/step
Epoch 6/30
121/121 - 86s - loss: 1.6492 - accuracy: 0.4073 - val_loss: 1.4340 - val_accuracy: 0.5374 - 86s/epoch - 711ms/step
Epoch 7/30
121/121 - 86s - loss: 1.5979 - accuracy: 0.4367 - val_loss: 1.3384 - val_accuracy: 0.5514 - 86s/epoch - 708ms/step
Epoch 8/30
121/121 - 85s - loss: 1.5006 - accuracy: 0.4588 - val_loss: 1.2053 - val_accuracy: 0.5888 - 85s/epoch - 706ms/step
Epoch 9/30
121/121 - 85s - loss: 1.4472 - accuracy: 0.4723 - val_loss: 1.1977 - val_accuracy: 0.5888 - 85s/epoch - 701ms/step
Epoch 10/30
121/121 - 85s - loss: 1.4010 - accuracy: 0.4965 - val_loss: 1.2344 - val_accuracy: 0.5631 - 85s/epoch - 701ms/step
Epoch 11/30
121/121 - 84s - loss: 1.3305 - accuracy: 0.5126 - val_loss: 1.1481 - val_accuracy: 0.6215 - 84s/epoch - 693ms/step
Epoch 12/30
121/121 - 83s - loss: 1.3096 - accuracy: 0.5090 - val_loss: 1.0901 - val_accuracy: 0.6028 - 83s/epoch - 685ms/step
Epoch 13/30
121/121 - 84s - loss: 1.2673 - accuracy: 0.5243 - val_loss: 1.1314 - val_accuracy: 0.6285 - 84s/epoch - 698ms/step
Epoch 14/30
121/121 - 84s - loss: 1.2262 - accuracy: 0.5422 - val_loss: 1.1081 - val_accuracy: 0.6402 - 84s/epoch - 695ms/step
Epoch 15/30
121/121 - 85s - loss: 1.2253 - accuracy: 0.5383 - val_loss: 1.0839 - val_accuracy: 0.6425 - 85s/epoch - 699ms/step
Epoch 16/30
121/121 - 86s - loss: 1.2199 - accuracy: 0.5487 - val_loss: 1.0152 - val_accuracy: 0.6682 - 86s/epoch - 713ms/step
Epoch 17/30
121/121 - 85s - loss: 1.1633 - accuracy: 0.5651 - val_loss: 0.9860 - val_accuracy: 0.6752 - 85s/epoch - 706ms/step
Epoch 18/30
121/121 - 87s - loss: 1.1059 - accuracy: 0.5810 - val_loss: 1.0212 - val_accuracy: 0.6963 - 87s/epoch - 719ms/step
Epoch 19/30
121/121 - 85s - loss: 1.1294 - accuracy: 0.5766 - val_loss: 0.9997 - val_accuracy: 0.6963 - 85s/epoch - 700ms/step
Epoch 20/30
121/121 - 86s - loss: 1.0791 - accuracy: 0.6038 - val_loss: 1.0237 - val_accuracy: 0.6799 - 86s/epoch - 713ms/step
Epoch 21/30
121/121 - 85s - loss: 1.0847 - accuracy: 0.5862 - val_loss: 0.9895 - val_accuracy: 0.6916 - 85s/epoch - 702ms/step
Epoch 22/30
121/121 - 87s - loss: 1.0690 - accuracy: 0.5966 - val_loss: 1.0153 - val_accuracy: 0.6822 - 87s/epoch - 719ms/step
Epoch 23/30
121/121 - 85s - loss: 1.0503 - accuracy: 0.5942 - val_loss: 1.0014 - val_accuracy: 0.6799 - 85s/epoch - 706ms/step
Epoch 24/30
121/121 - 87s - loss: 1.0473 - accuracy: 0.6015 - val_loss: 1.0492 - val_accuracy: 0.6822 - 87s/epoch - 718ms/step
Epoch 25/30
121/121 - 86s - loss: 1.0178 - accuracy: 0.6028 - val_loss: 1.0024 - val_accuracy: 0.6893 - 86s/epoch - 711ms/step
Epoch 26/30
121/121 - 88s - loss: 0.9643 - accuracy: 0.6319 - val_loss: 1.0023 - val_accuracy: 0.7009 - 88s/epoch - 724ms/step
Epoch 27/30
121/121 - 86s - loss: 0.9920 - accuracy: 0.6218 - val_loss: 1.0227 - val_accuracy: 0.7033 - 86s/epoch - 714ms/step
Epoch 28/30
121/121 - 87s - loss: 0.9528 - accuracy: 0.6374 - val_loss: 1.0558 - val_accuracy: 0.7079 - 87s/epoch - 720ms/step
Epoch 29/30
121/121 - 85s - loss: 0.9523 - accuracy: 0.6397 - val_loss: 0.9996 - val_accuracy: 0.7150 - 85s/epoch - 703ms/step
Epoch 30/30
121/121 - 85s - loss: 0.9798 - accuracy: 0.6241 - val_loss: 1.0287 - val_accuracy: 0.7150 - 85s/epoch - 705ms/step

Model evaluation

## plotting training and validation curve for above trained model
plt.plot(history_1.history['accuracy'])
plt.plot(history_1.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

We can see from above plot that our model has decent accuracy for both training and validation and our model seems to not to be overfitting which is a good sign but still we can see that the accuracy score for training and validation is plateauing around 70% for training curve and for validation curve it is reaching 60%, which should be increased to get better results

Evaluate model on test data

accuracy = model1.evaluate(X_test_normalized, y_test_encoded, verbose=2)    # Code to evaluate the model on test data

15/15 - 3s - loss: 0.9227 - accuracy: 0.7137 - 3s/epoch - 179ms/step

we can see from above output that accuracy for our first model is only 71.37% for test data , which is not good as only 71% of test rows are classified correctly which we should work on increasing to have a better model for prediction

Plotting the confusion matrix

# Here we would get the output as probablities for each category
y_pred=model1.predict(X_test_normalized)                          # code to predict the output probabilities

15/15 [==============================] - 3s 182ms/step

# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)

# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)              #  code to plot the confusion matrix
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
    confusion_matrix,
    annot=True,
    linewidths=.4,
    fmt="d",
    square=True,
    ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()

We can see from above confusion matrix that the most highest correctly predicted category is Loose silky bent species We can also see that there were no prediction for black grass in this model which is not a good sign lets see the classification report to learn more And we can also see that our model is getting confused between loose sikly-bent and black grass as we can see we predicted wrongly that it was loose sikly bent while it was blackgras

plotting classification report

from sklearn import metrics

# Plotting the classification report
cr=metrics.classification_report(y_test_arg,y_pred_arg)     #  code to plot the classification report
print(cr)

              precision    recall  f1-score   support

           0       0.00      0.00      0.00        26
           1       0.71      0.90      0.80        39
           2       0.50      0.31      0.38        29
           3       0.83      0.87      0.85        61
           4       0.75      0.14      0.23        22
           5       0.81      0.81      0.81        48
           6       0.58      0.95      0.73        65
           7       0.79      0.50      0.61        22
           8       0.67      0.79      0.73        52
           9       0.69      0.48      0.56        23
          10       0.95      0.84      0.89        50
          11       0.65      0.87      0.74        38

    accuracy                           0.71       475
   macro avg       0.66      0.62      0.61       475
weighted avg       0.69      0.71      0.68       475

We can see from above classification report that accuracy for this model on our test set is only 71% and we can also see that class 0 is not even predicted once , hence giving precision of 0 for about 26 count.

# Clearing backend
backend.clear_session()

Model Performance Improvement

Reducing the Learning Rate:

Hint: Use ReduceLRonPlateau() function that will be used to decrease the learning rate by some factor, if the loss is not decreasing for some time. This may start decreasing the loss at a smaller learning rate. There is a possibility that the loss may still not decrease. This may lead to executing the learning rate reduction again in an attempt to achieve a lower loss.

# Code to monitor val_accuracy
learning_rate_reduction = ReduceLROnPlateau(monitor='val_accuracy',
                                            patience=3,
                                            verbose=1,
                                            factor=0.5,
                                            min_lr=0.00001)

Data Augmentation

Remember, data augmentation should not be used in the validation/test data set.

# Clearing backend
from tensorflow.keras import backend
backend.clear_session()

# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)

# Code to set the rotation_range to 20
train_datagen = ImageDataGenerator(
                              rotation_range=20,
                              fill_mode='nearest'
                              )

# Intializing a sequential model
model2 = Sequential()

# code to add the first conv layer with 64 filters and kernel size 3x3 , padding 'same' provides the output size same as the input size
# Input_shape denotes input image dimension images
model2.add(Conv2D(64, (3, 3), activation='relu', padding="same", input_shape=(64, 64, 3)))

# code to add max pooling to reduce the size of output of first conv layer
model2.add(MaxPooling2D((2, 2), padding = 'same'))


model2.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model2.add(MaxPooling2D((2, 2), padding = 'same'))
model2.add(BatchNormalization())

# flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model2.add(Flatten())

# Adding a fully connected dense layer with 16 neurons
model2.add(Dense(16, activation='relu'))

#  code to add dropout with dropout_rate=0.3
model2.add(Dropout(0.3))
#  code to add the output layer with 12 neurons and activation functions as softmax since this is a multi-class classification problem
model2.add(Dense(12, activation='softmax'))

# code to initialize Adam Optimimzer
opt=Adam()
# code to Compile model
model2.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

# Generating the summary of the model
model2.summary()

Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d (Conv2D)             (None, 64, 64, 64)        1792      
                                                                 
 max_pooling2d (MaxPooling2  (None, 32, 32, 64)        0         
 D)                                                              
                                                                 
 conv2d_1 (Conv2D)           (None, 32, 32, 32)        18464     
                                                                 
 max_pooling2d_1 (MaxPoolin  (None, 16, 16, 32)        0         
 g2D)                                                            
                                                                 
 batch_normalization (Batch  (None, 16, 16, 32)        128       
 Normalization)                                                  
                                                                 
 flatten (Flatten)           (None, 8192)              0         
                                                                 
 dense (Dense)               (None, 16)                131088    
                                                                 
 dropout (Dropout)           (None, 16)                0         
                                                                 
 dense_1 (Dense)             (None, 12)                204       
                                                                 
=================================================================
Total params: 151676 (592.48 KB)
Trainable params: 151612 (592.23 KB)
Non-trainable params: 64 (256.00 Byte)
_________________________________________________________________

Fitting model on train data

#fit the model on train data with batch_size=64 and epochs=30
# Epochs
epochs = 30
# Batch size
batch_size = 64

history = model2.fit(train_datagen.flow(X_train_normalized,y_train_encoded,
                                       batch_size=batch_size,
                                       shuffle=False),
                                       epochs=epochs,
                                       steps_per_epoch=X_train_normalized.shape[0] // batch_size,
                                       validation_data=(X_val_normalized,y_val_encoded),
                                       verbose=1,callbacks=[learning_rate_reduction])

Epoch 1/30
60/60 [==============================] - 44s 697ms/step - loss: 2.1267 - accuracy: 0.2530 - val_loss: 2.3922 - val_accuracy: 0.1986 - lr: 0.0010
Epoch 2/30
60/60 [==============================] - 41s 687ms/step - loss: 1.6694 - accuracy: 0.4198 - val_loss: 2.3079 - val_accuracy: 0.2056 - lr: 0.0010
Epoch 3/30
60/60 [==============================] - 40s 665ms/step - loss: 1.4790 - accuracy: 0.4732 - val_loss: 2.2384 - val_accuracy: 0.3668 - lr: 0.0010
Epoch 4/30
60/60 [==============================] - 39s 655ms/step - loss: 1.3953 - accuracy: 0.5028 - val_loss: 2.1230 - val_accuracy: 0.5164 - lr: 0.0010
Epoch 5/30
60/60 [==============================] - 40s 665ms/step - loss: 1.2650 - accuracy: 0.5456 - val_loss: 1.8393 - val_accuracy: 0.4252 - lr: 0.0010
Epoch 6/30
60/60 [==============================] - 41s 680ms/step - loss: 1.1948 - accuracy: 0.5697 - val_loss: 1.6660 - val_accuracy: 0.5958 - lr: 0.0010
Epoch 7/30
60/60 [==============================] - 41s 675ms/step - loss: 1.1684 - accuracy: 0.5887 - val_loss: 1.8863 - val_accuracy: 0.3528 - lr: 0.0010
Epoch 8/30
60/60 [==============================] - 42s 696ms/step - loss: 1.1021 - accuracy: 0.6080 - val_loss: 1.1635 - val_accuracy: 0.6986 - lr: 0.0010
Epoch 9/30
60/60 [==============================] - 39s 648ms/step - loss: 1.0882 - accuracy: 0.6114 - val_loss: 1.1336 - val_accuracy: 0.6379 - lr: 0.0010
Epoch 10/30
60/60 [==============================] - 39s 645ms/step - loss: 1.0360 - accuracy: 0.6326 - val_loss: 0.9236 - val_accuracy: 0.7266 - lr: 0.0010
Epoch 11/30
60/60 [==============================] - 40s 654ms/step - loss: 1.0081 - accuracy: 0.6413 - val_loss: 1.0615 - val_accuracy: 0.6636 - lr: 0.0010
Epoch 12/30
60/60 [==============================] - 40s 673ms/step - loss: 0.9386 - accuracy: 0.6624 - val_loss: 2.2394 - val_accuracy: 0.3949 - lr: 0.0010
Epoch 13/30
60/60 [==============================] - ETA: 0s - loss: 0.9247 - accuracy: 0.6693
Epoch 13: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257.
60/60 [==============================] - 41s 677ms/step - loss: 0.9247 - accuracy: 0.6693 - val_loss: 0.8673 - val_accuracy: 0.7150 - lr: 0.0010
Epoch 14/30
60/60 [==============================] - 39s 653ms/step - loss: 0.8379 - accuracy: 0.6971 - val_loss: 0.7428 - val_accuracy: 0.7804 - lr: 5.0000e-04
Epoch 15/30
60/60 [==============================] - 39s 640ms/step - loss: 0.8227 - accuracy: 0.7031 - val_loss: 0.6984 - val_accuracy: 0.7850 - lr: 5.0000e-04
Epoch 16/30
60/60 [==============================] - 40s 658ms/step - loss: 0.8089 - accuracy: 0.7039 - val_loss: 0.7990 - val_accuracy: 0.7710 - lr: 5.0000e-04
Epoch 17/30
60/60 [==============================] - 40s 658ms/step - loss: 0.7831 - accuracy: 0.7090 - val_loss: 0.9160 - val_accuracy: 0.7290 - lr: 5.0000e-04
Epoch 18/30
60/60 [==============================] - ETA: 0s - loss: 0.7716 - accuracy: 0.7211
Epoch 18: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628.
60/60 [==============================] - 40s 659ms/step - loss: 0.7716 - accuracy: 0.7211 - val_loss: 0.9770 - val_accuracy: 0.6869 - lr: 5.0000e-04
Epoch 19/30
60/60 [==============================] - 40s 660ms/step - loss: 0.7404 - accuracy: 0.7261 - val_loss: 0.7440 - val_accuracy: 0.7664 - lr: 2.5000e-04
Epoch 20/30
60/60 [==============================] - 38s 638ms/step - loss: 0.7401 - accuracy: 0.7285 - val_loss: 0.6650 - val_accuracy: 0.8107 - lr: 2.5000e-04
Epoch 21/30
60/60 [==============================] - 40s 655ms/step - loss: 0.7260 - accuracy: 0.7351 - val_loss: 0.7375 - val_accuracy: 0.7850 - lr: 2.5000e-04
Epoch 22/30
60/60 [==============================] - 40s 668ms/step - loss: 0.7208 - accuracy: 0.7420 - val_loss: 0.7813 - val_accuracy: 0.7687 - lr: 2.5000e-04
Epoch 23/30
60/60 [==============================] - ETA: 0s - loss: 0.7200 - accuracy: 0.7304
Epoch 23: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814.
60/60 [==============================] - 40s 658ms/step - loss: 0.7200 - accuracy: 0.7304 - val_loss: 0.6634 - val_accuracy: 0.8037 - lr: 2.5000e-04
Epoch 24/30
60/60 [==============================] - 38s 633ms/step - loss: 0.7012 - accuracy: 0.7423 - val_loss: 0.6861 - val_accuracy: 0.8107 - lr: 1.2500e-04
Epoch 25/30
60/60 [==============================] - 39s 635ms/step - loss: 0.7101 - accuracy: 0.7375 - val_loss: 0.6610 - val_accuracy: 0.8178 - lr: 1.2500e-04
Epoch 26/30
60/60 [==============================] - 40s 650ms/step - loss: 0.6810 - accuracy: 0.7571 - val_loss: 0.6915 - val_accuracy: 0.8061 - lr: 1.2500e-04
Epoch 27/30
60/60 [==============================] - 42s 702ms/step - loss: 0.7109 - accuracy: 0.7431 - val_loss: 0.7735 - val_accuracy: 0.7734 - lr: 1.2500e-04
Epoch 28/30
60/60 [==============================] - ETA: 0s - loss: 0.6805 - accuracy: 0.7528
Epoch 28: ReduceLROnPlateau reducing learning rate to 6.25000029685907e-05.
60/60 [==============================] - 42s 701ms/step - loss: 0.6805 - accuracy: 0.7528 - val_loss: 0.6681 - val_accuracy: 0.8154 - lr: 1.2500e-04
Epoch 29/30
60/60 [==============================] - 40s 660ms/step - loss: 0.6798 - accuracy: 0.7510 - val_loss: 0.6889 - val_accuracy: 0.8084 - lr: 6.2500e-05
Epoch 30/30
60/60 [==============================] - 41s 688ms/step - loss: 0.6762 - accuracy: 0.7502 - val_loss: 0.7471 - val_accuracy: 0.7710 - lr: 6.2500e-05

Model Evaluation

## Code for model evaluation for trained model
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

We can see after our imporvements from model1 in this model2 , we can see that the accuracy curve has increased from being a little below 70 to above 70%, but it is again plateauing around above 70%. we can also see that both curve are almost together hence we can safely say that our model is not overfitting here

Evaluate on test data

accuracy = model2.evaluate(X_test_normalized, y_test_encoded, verbose=2)  # code to evaluate the model on test data

15/15 - 1s - loss: 0.8453 - accuracy: 0.7495 - 1s/epoch - 97ms/step

From above output we can see that accuracy for this improved model2 is 74.95% which is better than model 1 , hence we did make some improvements

plotting confusion matrix

#  code to obtain the output probabilities
y_pred=model2.predict(X_test_normalized)

15/15 [==============================] - 2s 90ms/step

# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)

# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)     # Complete the code to obatin the confusion matrix
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
    confusion_matrix,
    annot=True,
    linewidths=.4,
    fmt="d",
    square=True,
    ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()

From Above confusion matrix we can notice that unlike model1 , here we did make some prediction for class black grass and from which we made 4 correct prediction which is a good sign . And also we can see that even in this model we have made the most right prediction for loose silky bent species Also we can see the confusion that we had in our old model about losse silky bent being black grass , but here it has reduced which is a good progress

Printing classification report

# Plotting the classification report
cr=metrics.classification_report(y_test_arg,y_pred_arg)     #  code to plot the classification report
print(cr)

              precision    recall  f1-score   support

           0       0.44      0.27      0.33        26
           1       0.72      0.92      0.81        39
           2       0.63      0.59      0.61        29
           3       0.98      0.87      0.92        61
           4       0.78      0.32      0.45        22
           5       0.79      0.77      0.78        48
           6       0.64      0.78      0.70        65
           7       0.54      0.95      0.69        22
           8       0.77      0.79      0.78        52
           9       0.72      0.57      0.63        23
          10       0.91      0.84      0.87        50
          11       0.86      0.82      0.84        38

    accuracy                           0.75       475
   macro avg       0.73      0.71      0.70       475
weighted avg       0.76      0.75      0.74       475

We can see from above classification report that compared to older model we have made improvement in accuracy, now our accuracy is 75% , which is a good sign and also we can see that we have made some imporvement in predicting the class 0 that is black grass species

Hence as we are going in good direction lets try and further improve our model

Further Model Improvement , Lets now change our activation function to LeakyRelu and increasing convultion layer and pooling layer

# Clearing backend
from tensorflow.keras import backend
backend.clear_session()

# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)

from tensorflow.keras.layers import LeakyReLU

# Intializing a sequential model
model3 = Sequential()

# code to add the first conv layer with 64 filters and kernel size 3x3 , padding 'same' provides the output size same as the input size
# Input_shape denotes input image dimension images
model3.add(Conv2D(64, (3, 3), activation=LeakyReLU(), padding="same", input_shape=(64, 64, 3)))
model3.add(Conv2D(32, (3, 3), activation=LeakyReLU(), padding="same"))
#  code to add max pooling
model3.add(MaxPooling2D((2, 2), padding = 'same'))
model3.add(Conv2D(32, (3, 3), activation=LeakyReLU(), padding="same"))
model3.add(Conv2D(16, (3, 3), activation=LeakyReLU(), padding="same"))
model3.add(MaxPooling2D((2, 2), padding = 'same'))
model3.add(BatchNormalization())

# flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model3.add(Flatten())
model3.add(Dense(32, activation=LeakyReLU()))
# Adding a fully connected dense layer with 32 neurons
model3.add(Dense(16, activation=LeakyReLU()))
#  code to add dropout with dropout_rate=0.3
model3.add(Dropout(0.3))
# Code to add the output layer with 12 neurons and activation functions as softmax since this is a multi-class classification problem
model3.add(Dense(12, activation='softmax'))

# Code to initialize Adam Optimimzer
opt=Adam()
# Code to Compile model
model3.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

# Generating the summary of the model
model3.summary()

Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d (Conv2D)             (None, 64, 64, 64)        1792      
                                                                 
 conv2d_1 (Conv2D)           (None, 64, 64, 32)        18464     
                                                                 
 max_pooling2d (MaxPooling2  (None, 32, 32, 32)        0         
 D)                                                              
                                                                 
 conv2d_2 (Conv2D)           (None, 32, 32, 32)        9248      
                                                                 
 conv2d_3 (Conv2D)           (None, 32, 32, 16)        4624      
                                                                 
 max_pooling2d_1 (MaxPoolin  (None, 16, 16, 16)        0         
 g2D)                                                            
                                                                 
 batch_normalization (Batch  (None, 16, 16, 16)        64        
 Normalization)                                                  
                                                                 
 flatten (Flatten)           (None, 4096)              0         
                                                                 
 dense (Dense)               (None, 32)                131104    
                                                                 
 dense_1 (Dense)             (None, 16)                528       
                                                                 
 dropout (Dropout)           (None, 16)                0         
                                                                 
 dense_2 (Dense)             (None, 12)                204       
                                                                 
=================================================================
Total params: 166028 (648.55 KB)
Trainable params: 165996 (648.42 KB)
Non-trainable params: 32 (128.00 Byte)
_________________________________________________________________

#fit the model on train data
# Epochs
epochs = 60
# Batch size
batch_size = 128

history3 = model3.fit(train_datagen.flow(X_train_normalized,y_train_encoded,
                                       batch_size=batch_size,
                                       shuffle=False),
                                       epochs=epochs,
                                       steps_per_epoch=X_train_normalized.shape[0] // batch_size,
                                       validation_data=(X_val_normalized,y_val_encoded),
                                       verbose=1,callbacks=[learning_rate_reduction])

Epoch 1/60
30/30 [==============================] - 99s 3s/step - loss: 2.1571 - accuracy: 0.2675 - val_loss: 2.4092 - val_accuracy: 0.2383 - lr: 0.0010
Epoch 2/60
30/30 [==============================] - 95s 3s/step - loss: 1.7014 - accuracy: 0.4434 - val_loss: 2.3741 - val_accuracy: 0.1425 - lr: 0.0010
Epoch 3/60
30/30 [==============================] - 95s 3s/step - loss: 1.4678 - accuracy: 0.5187 - val_loss: 2.2891 - val_accuracy: 0.2103 - lr: 0.0010
Epoch 4/60
30/30 [==============================] - 93s 3s/step - loss: 1.3579 - accuracy: 0.5380 - val_loss: 2.2385 - val_accuracy: 0.3178 - lr: 0.0010
Epoch 5/60
30/30 [==============================] - 95s 3s/step - loss: 1.1934 - accuracy: 0.6058 - val_loss: 2.1776 - val_accuracy: 0.2967 - lr: 0.0010
Epoch 6/60
30/30 [==============================] - 94s 3s/step - loss: 1.1123 - accuracy: 0.6308 - val_loss: 2.0569 - val_accuracy: 0.4509 - lr: 0.0010
Epoch 7/60
30/30 [==============================] - 97s 3s/step - loss: 1.0249 - accuracy: 0.6628 - val_loss: 1.9068 - val_accuracy: 0.5164 - lr: 0.0010
Epoch 8/60
30/30 [==============================] - 97s 3s/step - loss: 0.9280 - accuracy: 0.6967 - val_loss: 1.8146 - val_accuracy: 0.5047 - lr: 0.0010
Epoch 9/60
30/30 [==============================] - 96s 3s/step - loss: 0.8819 - accuracy: 0.6956 - val_loss: 1.6058 - val_accuracy: 0.4673 - lr: 0.0010
Epoch 10/60
30/30 [==============================] - 98s 3s/step - loss: 0.8210 - accuracy: 0.7228 - val_loss: 1.4382 - val_accuracy: 0.5818 - lr: 0.0010
Epoch 11/60
30/30 [==============================] - 97s 3s/step - loss: 0.7680 - accuracy: 0.7443 - val_loss: 1.3259 - val_accuracy: 0.6449 - lr: 0.0010
Epoch 12/60
30/30 [==============================] - 98s 3s/step - loss: 0.7623 - accuracy: 0.7462 - val_loss: 1.1314 - val_accuracy: 0.6986 - lr: 0.0010
Epoch 13/60
30/30 [==============================] - 96s 3s/step - loss: 0.6905 - accuracy: 0.7634 - val_loss: 1.4255 - val_accuracy: 0.5257 - lr: 0.0010
Epoch 14/60
30/30 [==============================] - 98s 3s/step - loss: 0.6674 - accuracy: 0.7704 - val_loss: 0.8170 - val_accuracy: 0.7687 - lr: 0.0010
Epoch 15/60
30/30 [==============================] - 95s 3s/step - loss: 0.6063 - accuracy: 0.7916 - val_loss: 0.8429 - val_accuracy: 0.7383 - lr: 0.0010
Epoch 16/60
30/30 [==============================] - 98s 3s/step - loss: 0.5740 - accuracy: 0.8021 - val_loss: 0.7732 - val_accuracy: 0.7360 - lr: 0.0010
Epoch 17/60
30/30 [==============================] - ETA: 0s - loss: 0.6000 - accuracy: 0.7956
Epoch 17: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257.
30/30 [==============================] - 96s 3s/step - loss: 0.6000 - accuracy: 0.7956 - val_loss: 0.7630 - val_accuracy: 0.7453 - lr: 0.0010
Epoch 18/60
30/30 [==============================] - 97s 3s/step - loss: 0.5234 - accuracy: 0.8217 - val_loss: 0.7135 - val_accuracy: 0.7734 - lr: 5.0000e-04
Epoch 19/60
30/30 [==============================] - 96s 3s/step - loss: 0.4714 - accuracy: 0.8430 - val_loss: 0.6217 - val_accuracy: 0.7944 - lr: 5.0000e-04
Epoch 20/60
30/30 [==============================] - 96s 3s/step - loss: 0.4462 - accuracy: 0.8443 - val_loss: 0.7277 - val_accuracy: 0.7313 - lr: 5.0000e-04
Epoch 21/60
30/30 [==============================] - 100s 3s/step - loss: 0.4363 - accuracy: 0.8532 - val_loss: 0.5753 - val_accuracy: 0.8271 - lr: 5.0000e-04
Epoch 22/60
30/30 [==============================] - 100s 3s/step - loss: 0.4199 - accuracy: 0.8572 - val_loss: 0.6097 - val_accuracy: 0.8084 - lr: 5.0000e-04
Epoch 23/60
30/30 [==============================] - 99s 3s/step - loss: 0.4052 - accuracy: 0.8623 - val_loss: 1.1624 - val_accuracy: 0.6589 - lr: 5.0000e-04
Epoch 24/60
30/30 [==============================] - ETA: 0s - loss: 0.4117 - accuracy: 0.8532
Epoch 24: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628.
30/30 [==============================] - 96s 3s/step - loss: 0.4117 - accuracy: 0.8532 - val_loss: 0.6558 - val_accuracy: 0.7687 - lr: 5.0000e-04
Epoch 25/60
30/30 [==============================] - 97s 3s/step - loss: 0.3625 - accuracy: 0.8728 - val_loss: 0.6352 - val_accuracy: 0.8154 - lr: 2.5000e-04
Epoch 26/60
30/30 [==============================] - 95s 3s/step - loss: 0.3615 - accuracy: 0.8742 - val_loss: 0.5960 - val_accuracy: 0.8131 - lr: 2.5000e-04
Epoch 27/60
30/30 [==============================] - 97s 3s/step - loss: 0.3481 - accuracy: 0.8795 - val_loss: 0.5334 - val_accuracy: 0.8435 - lr: 2.5000e-04
Epoch 28/60
30/30 [==============================] - 97s 3s/step - loss: 0.3347 - accuracy: 0.8814 - val_loss: 0.6073 - val_accuracy: 0.8084 - lr: 2.5000e-04
Epoch 29/60
30/30 [==============================] - 96s 3s/step - loss: 0.3363 - accuracy: 0.8812 - val_loss: 0.5602 - val_accuracy: 0.8388 - lr: 2.5000e-04
Epoch 30/60
30/30 [==============================] - ETA: 0s - loss: 0.3300 - accuracy: 0.8849
Epoch 30: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814.
30/30 [==============================] - 94s 3s/step - loss: 0.3300 - accuracy: 0.8849 - val_loss: 0.7459 - val_accuracy: 0.7734 - lr: 2.5000e-04
Epoch 31/60
30/30 [==============================] - 98s 3s/step - loss: 0.3134 - accuracy: 0.8908 - val_loss: 0.5302 - val_accuracy: 0.8458 - lr: 1.2500e-04
Epoch 32/60
30/30 [==============================] - 98s 3s/step - loss: 0.3141 - accuracy: 0.8898 - val_loss: 0.5548 - val_accuracy: 0.8294 - lr: 1.2500e-04
Epoch 33/60
30/30 [==============================] - 97s 3s/step - loss: 0.3061 - accuracy: 0.8959 - val_loss: 0.5207 - val_accuracy: 0.8411 - lr: 1.2500e-04
Epoch 34/60
30/30 [==============================] - 98s 3s/step - loss: 0.3111 - accuracy: 0.8924 - val_loss: 0.5370 - val_accuracy: 0.8598 - lr: 1.2500e-04
Epoch 35/60
30/30 [==============================] - 99s 3s/step - loss: 0.3090 - accuracy: 0.8935 - val_loss: 0.6334 - val_accuracy: 0.8178 - lr: 1.2500e-04
Epoch 36/60
30/30 [==============================] - 97s 3s/step - loss: 0.2911 - accuracy: 0.9008 - val_loss: 0.5320 - val_accuracy: 0.8318 - lr: 1.2500e-04
Epoch 37/60
30/30 [==============================] - ETA: 0s - loss: 0.3116 - accuracy: 0.8919
Epoch 37: ReduceLROnPlateau reducing learning rate to 6.25000029685907e-05.
30/30 [==============================] - 98s 3s/step - loss: 0.3116 - accuracy: 0.8919 - val_loss: 0.5864 - val_accuracy: 0.8037 - lr: 1.2500e-04
Epoch 38/60
30/30 [==============================] - 93s 3s/step - loss: 0.2825 - accuracy: 0.9021 - val_loss: 0.5434 - val_accuracy: 0.8388 - lr: 6.2500e-05
Epoch 39/60
30/30 [==============================] - 98s 3s/step - loss: 0.2922 - accuracy: 0.9013 - val_loss: 0.5394 - val_accuracy: 0.8388 - lr: 6.2500e-05
Epoch 40/60
30/30 [==============================] - ETA: 0s - loss: 0.2840 - accuracy: 0.9019
Epoch 40: ReduceLROnPlateau reducing learning rate to 3.125000148429535e-05.
30/30 [==============================] - 96s 3s/step - loss: 0.2840 - accuracy: 0.9019 - val_loss: 0.5113 - val_accuracy: 0.8528 - lr: 6.2500e-05
Epoch 41/60
30/30 [==============================] - 98s 3s/step - loss: 0.2821 - accuracy: 0.9010 - val_loss: 0.5292 - val_accuracy: 0.8364 - lr: 3.1250e-05
Epoch 42/60
30/30 [==============================] - 97s 3s/step - loss: 0.2759 - accuracy: 0.9070 - val_loss: 0.5225 - val_accuracy: 0.8388 - lr: 3.1250e-05
Epoch 43/60
30/30 [==============================] - ETA: 0s - loss: 0.2859 - accuracy: 0.9035
Epoch 43: ReduceLROnPlateau reducing learning rate to 1.5625000742147677e-05.
30/30 [==============================] - 94s 3s/step - loss: 0.2859 - accuracy: 0.9035 - val_loss: 0.5176 - val_accuracy: 0.8481 - lr: 3.1250e-05
Epoch 44/60
30/30 [==============================] - 95s 3s/step - loss: 0.2863 - accuracy: 0.9040 - val_loss: 0.5153 - val_accuracy: 0.8435 - lr: 1.5625e-05
Epoch 45/60
30/30 [==============================] - 95s 3s/step - loss: 0.2817 - accuracy: 0.9013 - val_loss: 0.5220 - val_accuracy: 0.8388 - lr: 1.5625e-05
Epoch 46/60
30/30 [==============================] - ETA: 0s - loss: 0.2825 - accuracy: 0.9021
Epoch 46: ReduceLROnPlateau reducing learning rate to 1e-05.
30/30 [==============================] - 96s 3s/step - loss: 0.2825 - accuracy: 0.9021 - val_loss: 0.5140 - val_accuracy: 0.8481 - lr: 1.5625e-05
Epoch 47/60
30/30 [==============================] - 98s 3s/step - loss: 0.2739 - accuracy: 0.9043 - val_loss: 0.5148 - val_accuracy: 0.8481 - lr: 1.0000e-05
Epoch 48/60
30/30 [==============================] - 96s 3s/step - loss: 0.2624 - accuracy: 0.9113 - val_loss: 0.5194 - val_accuracy: 0.8388 - lr: 1.0000e-05
Epoch 49/60
30/30 [==============================] - 97s 3s/step - loss: 0.2773 - accuracy: 0.9099 - val_loss: 0.5183 - val_accuracy: 0.8458 - lr: 1.0000e-05
Epoch 50/60
30/30 [==============================] - 98s 3s/step - loss: 0.2804 - accuracy: 0.9027 - val_loss: 0.5169 - val_accuracy: 0.8481 - lr: 1.0000e-05
Epoch 51/60
30/30 [==============================] - 99s 3s/step - loss: 0.2729 - accuracy: 0.9056 - val_loss: 0.5188 - val_accuracy: 0.8435 - lr: 1.0000e-05
Epoch 52/60
30/30 [==============================] - 99s 3s/step - loss: 0.2880 - accuracy: 0.9005 - val_loss: 0.5135 - val_accuracy: 0.8505 - lr: 1.0000e-05
Epoch 53/60
30/30 [==============================] - 98s 3s/step - loss: 0.2608 - accuracy: 0.9059 - val_loss: 0.5112 - val_accuracy: 0.8528 - lr: 1.0000e-05
Epoch 54/60
30/30 [==============================] - 98s 3s/step - loss: 0.2739 - accuracy: 0.9083 - val_loss: 0.5148 - val_accuracy: 0.8435 - lr: 1.0000e-05
Epoch 55/60
30/30 [==============================] - 97s 3s/step - loss: 0.2707 - accuracy: 0.9067 - val_loss: 0.5158 - val_accuracy: 0.8435 - lr: 1.0000e-05
Epoch 56/60
30/30 [==============================] - 94s 3s/step - loss: 0.2802 - accuracy: 0.9045 - val_loss: 0.5188 - val_accuracy: 0.8388 - lr: 1.0000e-05
Epoch 57/60
30/30 [==============================] - 95s 3s/step - loss: 0.2700 - accuracy: 0.9040 - val_loss: 0.5176 - val_accuracy: 0.8481 - lr: 1.0000e-05
Epoch 58/60
30/30 [==============================] - 97s 3s/step - loss: 0.2857 - accuracy: 0.9024 - val_loss: 0.5155 - val_accuracy: 0.8411 - lr: 1.0000e-05
Epoch 59/60
30/30 [==============================] - 99s 3s/step - loss: 0.2664 - accuracy: 0.9072 - val_loss: 0.5191 - val_accuracy: 0.8435 - lr: 1.0000e-05
Epoch 60/60
30/30 [==============================] - 98s 3s/step - loss: 0.2687 - accuracy: 0.9051 - val_loss: 0.5218 - val_accuracy: 0.8411 - lr: 1.0000e-05

Model Evaluation

## code for model evaluation
plt.plot(history3.history['accuracy'])
plt.plot(history3.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

We can see from above accuracy curve plot for training and validation that our model is not overfitting and we have made some good improvement from previous models as our previous model had accuracy around 73 , but now our accuracy curve are around 80% for validation and around 90% for training

Evaluate on test data

accuracy = model3.evaluate(X_test_normalized, y_test_encoded, verbose=2)  # code to evaluate the model on test data

15/15 - 2s - loss: 0.5494 - accuracy: 0.8379 - 2s/epoch - 151ms/step

We can see from above output of our code that accuracy of our model on test data has increased to 83.79% which is a good sign and hence we have achieved good improvement

plotting confusion matrix

#  to obtain the output probabilities
y_pred=model3.predict(X_test_normalized)

15/15 [==============================] - 4s 256ms/step

# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)

# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)     # code to obatin the confusion matrix
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
    confusion_matrix,
    annot=True,
    linewidths=.4,
    fmt="d",
    square=True,
    ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()

From above confusion matrix we can see that now we are even more capable of predicting the black grass class which in previous models we were suffering to . and now along with loose silky bent our predictrion power for other class has also increased

Printing classification report

# Plotting the classification report
cr=metrics.classification_report(y_test_arg,y_pred_arg)     # Complete the code to plot the classification report
print(cr)

              precision    recall  f1-score   support

           0       0.58      0.42      0.49        26
           1       0.86      0.95      0.90        39
           2       0.78      0.86      0.82        29
           3       0.93      0.90      0.92        61
           4       0.79      0.68      0.73        22
           5       0.93      0.88      0.90        48
           6       0.76      0.82      0.79        65
           7       0.72      0.82      0.77        22
           8       0.78      0.96      0.86        52
           9       0.88      0.61      0.72        23
          10       0.96      0.90      0.93        50
          11       0.92      0.87      0.89        38

    accuracy                           0.84       475
   macro avg       0.82      0.81      0.81       475
weighted avg       0.84      0.84      0.83       475

We can see from above classification report that our overall scores are improved compared to older models and we have achieved a accuracy score of 84% for test data using this model3

Final Model

Hence as we have increased power of predicting difficult class like black grass in model3 and also we have acheived a good accuracy score of 84% for test data in model3 hence we are selecting model3 as our final model

We can see that model 3 is the model that we are selecting as our final model. As model 3 has did a better job than the previous two models in identifying and classifing the plant species.

Visualizing the prediction

# Visualizing the predicted and correct label of images from test data
plt.figure(figsize=(2,2))
plt.imshow(X_test[2])
plt.show()
## Complete the code to predict the test data using the final model selected
print('Predicted Label', enc.inverse_transform(model3.predict((X_test_normalized[2].reshape(1,64,64,3)))))   # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[2])                                               # using inverse_transform() to get the output label from the output vector

plt.figure(figsize=(2,2))
plt.imshow(X_test[33])
plt.show()
## Complete the code to predict the test data using the final model selected
print('Predicted Label', enc.inverse_transform(model3.predict((X_test_normalized[33].reshape(1,64,64,3)))))  # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[33])                                              # using inverse_transform() to get the output label from the output vector

plt.figure(figsize=(2,2))
plt.imshow(X_test[59],)
plt.show()
## Complete the code to predict the test data using the final model selected
print('Predicted Label', enc.inverse_transform(model3.predict((X_test_normalized[59].reshape(1,64,64,3)))))  # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[59])                                              # using inverse_transform() to get the output label from the output vector

plt.figure(figsize=(2,2))
plt.imshow(X_test[36])
plt.show()
## Complete the code to predict the test data using the final model selected
print('Predicted Label', enc.inverse_transform(model3.predict((X_test_normalized[36].reshape(1,64,64,3)))))  # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[36])                                              # using inverse_transform() to get the output label from the output vector

1/1 [==============================] - 0s 35ms/step
Predicted Label ['Small-flowered Cranesbill']
True Label Small-flowered Cranesbill

1/1 [==============================] - 0s 31ms/step
Predicted Label ['Cleavers']
True Label Cleavers

1/1 [==============================] - 0s 27ms/step
Predicted Label ['Common Chickweed']
True Label Common Chickweed

1/1 [==============================] - 0s 31ms/step
Predicted Label ['Shepherds Purse']
True Label Shepherds Purse

Actionable Insights and Business Recommendations

• Data Collection and Quality:

1. Expand Dataset: Gather additional data, especially for classes with fewer samples, to enhance model accuracy and robustness. Increasing the diversity and quantity of data can help the model generalize better to unseen examples.

2. Improve Image Capture: Advise improvements in the image capture process to ensure high-quality images across all classes. Consistency in lighting conditions, angles, and backgrounds can mitigate variability and improve model performance.

3. Comprehensive Dataset: Encourage the creation of a comprehensive dataset with a broader range of images per class, including different growth stages and environmental conditions. This variability can enrich the model's understanding of plant characteristics.

• Application and Integration:

1. Mobile Applications: Propose the development of mobile applications or embedded systems utilizing the model for real-time plant seedling classification. Enabling farmers to identify and manage weeds on the spot can enhance agricultural productivity and efficiency.

2. Drone Integration: Suggest integrating the model with drone technology for efficient monitoring of large agricultural fields. Drones equipped with the model can swiftly identify plant species across vast areas, aiding in precision agriculture and resource optimization.

• Further Research and Experimentation:

1. Transfer Learning: Recommend exploring transfer learning techniques with pre-trained models to potentially improve performance or efficiency, especially in resource-constrained environments. Leveraging pre-trained models can expedite training and enhance model adaptability.

2. Semi-supervised Learning: Investigate semi-supervised or unsupervised learning approaches to utilize unlabeled data effectively. This can be particularly beneficial in scenarios where labeled data is scarce or expensive to obtain.

• Broader Implications and Future Directions:

1. Scale to More Species: Discuss the potential for scaling the solution to classify additional plant species beyond the current dataset. Expanding the model's capability to identify a broader range of plants can further enhance its utility and applicability in diverse agricultural settings.

2. Sustainable Agriculture: Highlight the broader implications of AI in agriculture for promoting sustainable practices. Emphasize how precise weed identification facilitated by AI can contribute to reducing herbicide usage, thereby promoting environmentally-friendly farming practices and biodiversity conservation.

These recommendations and insights can guide the development and deployment of AI-based solutions in agriculture, fostering innovation and sustainability in the industry.

---